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suumary is pfesented of an intensive assessoent of 
the 'impact of a two-year effort tc assist 30 colleges and, 
uni varsities to improve their faculty evaluatiOTi procedures. The 
Southera Regional Educate or. Board (SETBI ,« supported by a g^nt frOia 
the Fund for the loprovemer.t o^ Postsesoniary Education/ work^ 
closely with teans of faculty arid adafinistrators in worjcsKop and ^ 
consultation s<^ttings '♦jo help them design evaluation approaches i<hicb>^ 
were sore coaprehensive ?.nd pystematic. The .vast majority of tha 
institutions eiade s ign-if ican* proqress toward accoEpliffhing thei: 
goals, in general, institutions wl*h the most ambitious goals isiie 
the greatest progresp. Sever charactaristics, most r.otabl y supper t 
ani iavDlvement by tDp-level admirif^t retors, influenced the progress 
and probability of l=stincr impact. Full ^nd extensive faculty 
icvoivaiaant also proved ef=;5?er.* ial. Certain proiect activities, in , 
particular the campus visit*? bv proiect- sponsored consultants, proved 
siqni£i;:antly helpful. Par+'icipar.t s? learned several things about the 
politics of change, as well as ?.bout facult\^ evalution. A, regional, 
intsriastitutional approach provided valuable stiatulatioii and access 
to ei^pert knowledge. fRuthor/CTHI ; 
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This report is a. suima4tlye evaluation of the faculty Evaluation Project ad&iinistered 
by the Southern Regiondl fi^ucation 'Bo^ard (SREB) with grant suppprt from the Fund for 
Improvement of Pp^tsepoudar^ Education (FIPS^ It has beeii prepared the project' 
three-^member evaluation teamyrand is presented in three parts: 1) a description 
of the overall design and data collection procedures employed; 2) a summary of 
the evaluation findings! and 3) concli^si^ns reached about the success' of faculty * 
evaluation strategies in genera^l, and project impacts in particular, 

; •< . PART I: EVALUATION DESIGN 



Early in the project (August, 1977), the evaluation team completed a comprehensive 
design for formative* and summative evaluation of the project. As a result of 
leagthy team deliberations, and extensive discussion with SREB staff, a 'list of 
ten major criterion areas eventually emerged., Taken together, these evaluation 
questions provided a focus for the project assessment reported here. The ten 
criterion areas are couched in terms of the following questions. 

1. Has progress been made toward an lisp roved faculty evaluatiou systeSr? What 
have been the chief positive and negative influences? 

2, To what. extent have -project goals been achieved? How have expectation^ - 
changed? UTiat have been the major surprises? 0. 

3. Who has been involved in the project? What has been the nature of their 
involvement? - 

4, How useful is the "team approach'* in this kind of endeavor? How are teatn 
members viewed on campiis? 



5. How has SREB helped or hindered progress? Where would the institution be 
without participation in the project? 

6. Are there any project "spinoffs" on campus (e.g., faculty development 
activities, contacts with other schools. Increased canpus visibility)? 

7. What is the campus "xqood" with regard to faculty evaluation? By' what 
factions are these feelings held? What accous^ts for them? 

* ■ ............ . , , 

8. What is the likelihood of, permanent changes in faculty evaluation procedures? 
What has to happen to insure such changes? 

* 9« What has been learned atn^ut faculty evaluation? Its relationship to 

faculty de- 'elopment ? To student learning? 
10. Have FlPSE's and SREB' ^Investments been worthwhile overall? - 

In order to answer these questions > evaluative data were needed from a . 

« 

variety of sources. Specifically, the * evaluation design called for the following 
1. , Evaluation of each project activity (cDnferenc,es, consultation visits, eto.)' 

by participants, to determine how well the activity accomplished its , 

objectives; . • 

r 

^. Development of a "portfolio" by team meiabers at each institution, containing 
all relevant documentation relating to progress toward institutional (and 
project) goals; 

3» Data collected by evaluation team members themselves; conference yaluations 
.interviews with project team members, SREB staff visits to participating 
Institutions, and evaluation site visit reports; and 
A. 'Interim and final progress reports from each institution in the project. 



During the course of the project » the evaluation team has heea intricately 
involved (on d rotating basis) in the evaluation of each project activity, and 
has provided routine feedback through reports to SEEB staff. In addition, 
members of the evaluation team have maintained an on-going, informal contact 
/wl^ SREB staff and have met with the project Task Force (once each year) to 
provide formative and sumuitive infomatlon on the pTOject. 

. ■ ♦ ■ 

Data Collection Instruments 

Semt'-annual ^forkshops ^ On a rotating hasis^ tvo members of the evaluation , 
team attended each of the project-sponsored workshops. Attendance at these 
meetings allowed the evaluation team to observe the level of interest generated 
in the project, to report the nature and scope of evaluation activities to project • 
participants, and to assess the relevance of vt^rkshop materials and activities. 

evaluation form was developed by the evaluation team to assess the effectiveness 
of these «R>rkshops. 

Interviewa with representatives from participating institutions . In con-, 
^iunction with the semi-annual workshops, the evaluation team conducted interviews 
' with a representative ('.'contact person") from each participating' institution during 
the second year of the project*. An interview schedule was developed by the 
evaluation team and Interview reports wer.e shared with SREB staff. 

• Evaluation of consultant visits . Following each visit by an SREB-sponsorcd 
consultant to one of the 30 project schools, both the consultant and campus team 
members completed an evaluation form. The consultant wSs asked to indicate and 
rate accomplishment of personal objectives, note critical events, and evaluate 



the team's progress. Team nembera were, askeid to indicate and rate accomplishment 
of their objectives, note critl<*al eventf, and rate the consultant's skills and 
helpfulness.' 

■. I • • • 

Samples of evaluation forms us^ to collect all of the above data are pro~ 

vided in the Appendix. 

Site visits . During the final months of the project, the evaluation team 
conducted site viaits at 15 institutions. Each evaluator visited five colleges. 
Since stratified randcm sampling procedures were employed, the jUistltuticms 
selected were diverse, not only in size and location, but also in the types of 
faculty evaluation procedures undergplng development. The table below shows 
scKoe of this diversity. 

Institution Public Private ^otal 

Two-year 4 1 5 

Baccalaureate 2 1 3 

blasters 2 . 2 4 

Doctoral - 3 , ' 0 3 



,. ^^^^^ . ^ 
%1 . . TOTAL 11 4 I 15 



purposes of these site visits were to get a first-hand view of projects on 
x^lBnp^s and to discuss project activities with administrators, project team 
members, faculty leaders, and other faculty members. 

The casipus visits lasted one day each. In preparation for each visit the 
evaluator read the college's portfolio supplied by SREB. These portfolios 
contained the following pieces of information: 1) institutional background 
information, including the nature of the institution's faculty evaluation 

7 



pplldes and procedures prior to the Faculty BvaXuation Proiect, 2) consultant 
and workshop evaluations by the caapus teaa aeidters and the consultants t 
3} progress reports, and 4> information on other critical events and activities 
that had occurred throv^out the project. The evaluator's actual on-site 
activity consisted of a series of interviews » including in inost cases couver- 
sations with each of the following: 

The SREB campus te^ . * . 

The chief academic officer. ^ 
Hhe president or chancellor * 
A group of three or four faculty leaders (e»g.. Faculty Senate President, 

Chairman of Promotion and Tenure Committee, Union President* etc) 
A group of four or five faculty timbers representing **opinion leaders" 

among the faculty but unconnected with the project. 

In order to permit attention to si^ficant teterinstitutiottal differences » 
no standard form was developed in advance for use with each interview. Questions 
relating to each of the ten major criterion areas were asked throughout the 
day, however, and site visit reports followed a format consistent with these 
criteria. Site visit reports were shared with the on-cai^xu team leaders for 

s * 

their reaction and correction of factual errors. In the few cases where a dispute 
over interpretation arose, further discussion Occurred between the ev^lustor and 
the cai^us team in an attest to resolve their differences. 

Preparation ci Final Evaluation Report 

Prior to the evaluatiwa team*s final meeting in April, 1979, each evaluation 
team member reviewed five additional SREB portfolios on colleges that had not 
been selected for evaluation site visits. This maat t^t each evaluator had 
a fairly good grasp of the impact of the project on ten different colleges when 
the team s^t to prepare this summative evaluation. (In addition to reviewing' ' 
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each college's portfolio ptioir to, the ^ril the evaluation .te«ffi re- 

viewed suffloaries of the int^rvijews thit^yere held during the July 1978 and 
January 1979 workshops.) 

By the time 6£ the ^ril 1979 meeting, the evaluation team as a whole was 
very faaiiliar vith the nature and extent p£ activity on each of ^he 30 project 
college casspusfti. In that ^ril meeting, the evaluation teas 1) reviewed Its 
perceptions of the intact of the project at each college with SREB staff* 
2} developed a t«7o or three sentence statement describing progress and goal 

..... , -Til- ' 

achievement at each Icollege, 3) identified the majoip positive and negative 

* 

influences for constructive change^ on each caopus^ ^X reviewed SREB's role in 
the project, and S) ideatlfied colleges that had aj^ready achieved some permanence 
(in^lemented new policies and/or procedures in,1978*-79)t colleges where penaanent 
changes in faculty evaluation policies and procedures apj^ptred likely in another 
year or two, and colleges where change in the foreseeable future seeioed unlikely*. 
The evaluation results and conclusioris presented in the next two sections are 
based on these discussions* 



Evaluation findings nee reporUd here la five parts: a) progress and pul * ' 
acbievenesit; -h}/ prospects for pariaaaeiit Sxap&ct of new or revised faculty evalu- 
atioa procedures oa campiisj c) major* factors responsible for progress Cor lack 
thereof); d) the role sad influence of SREB; and e) lessons learned by the 30 
colleges as a function of project participation. 13ieae five areas represent a 
distillation of the ten major criterion categories eoplo/ed in the overall 
project evaluation plan. 

# 

ProBrcss and Goal Achievement 

^ 30 participating Institutions developed a wide variety of goals, dependent in 
large measure upon the status of their faculty evaluation prograsi at the beginninL* 
of the project. In general, institijitlonal goals for faculty evaluation fell info 
three categories. Fifteen institutions had the amjbltious goal of developing a new 
comprehensive faculty evaluation system from scratch. Sine others, who had alrfeady 
adopted systematic procedures in some form, planned to modify or "fine tune" th^ir 
current system. Finally, six colleges aimed ta review and assess the status quo , 
increase communication about faculty evaluation within the cacqjus community, and 
develop more consistent policies and procedures. The larger institutions in 
the pTojeJt tended t6 be assscg the scfiools in this final category. 

a. Of the fifteen Itidtitutlons attesspting to develop coaiyrel^ttsive systems t prepress 

has been variables * ?ive have accomplished their goals in fulls i.e* • a new s^FStem has 

bean developed, field tested, approved, add readied, for full Itspleoentations Four have 
♦ 

developed a new systea^that is currently being pilot-tested; four have developed parts 

in ■' 



o£ a syscen (such as a new student satliis fon&)i md tvo have not progressed far 

.• • ..... -. ■ ■ \. ^ ■' . ... yf-' ' 

beyond preliuinary data coliectipn* such as faculty survays md intervi«fis. . 

b* Of tha nine institutiooe aioicg to "fine tune*' their current ptocaduiras* 
fiigalf leant progress has >eeti made In all but one » although along somewhat different 
lines. In several of these schools the main focus of attention has been the develop- 
Bent of a revised studeab rating form; others have concentrated on tying their, systca 
more closely to faculty developa^t; still others have used the time to stt^y and 
pilot test their procedures and gain greater faculty acceptance for the system. Th? 
one school in this group denonstrating a notable lack of progress has suffered frota 
• pqor cooBBunication between faculty and the ad^tinist ration, resulting In suspicion 

about how the results are to be used. 

■ ■ . ■ ' \ ' 

c. In the eix institutions focusing on review of policy, variable progress has 

! r ■ ■ ■ ■ 

. ... . . ■ 5 

been made. As previously noted, these tended to be the larger schools; only one In 
a conuounlty college, one Is a masters-level Institution, and all the rest are xmi jc^c- 
sltles. Thus, the development of more consistent policies and adoption of mere 
standardized cas^us-wide Instruments is a s^re asA^ltious t^k than may at first ^ 
'appear. One of thp universities conducted an exh«iustive survey of faculty 3 . 
administrators resulting in major policy changes; at another, a proposed plan for 
evaluation is currently being considered by the faculty and a new student rating 
instrufflcnt has been adopted by five of Its. six schools; at another, unionized 
institution, a new\proB»tiop «»nd tenure statement has been proposed; and two sclwola 
(including the junior college) have developed new student rating forms. At the sixth 
institution the project team has played a very low-key role by consulting with 
individual departments, with little obiserv^leHropact , 

In summary, then, with a few exceptions, the institutional teams have oade slgnlflrnj^c 
progress toward accomplishiivg their original goals* This progress has perhaps been 



most iotpressivQ in ^ho6e colleges in the first group who started froo •'ground- zero" s V 
for live oC Cifteen schools /to ^ave developed and isspleniented a hrand new coi^prehenslve 

Drstem of faculty evaliiatloa Iti l#ss thaa eighteen, noaths* timet and for jtU l»ut two 

' . ^ • • • • ... 49 - ' * \ - 

of the other ten to have wade significant inroads during this period lis ^ a significant 
aecoi&pliahment by an y standards Further^ there are s^j or .successes in both of the ' 
other two groups as well. Overall , across nfpjlct tnatttutions. obaetvable 

♦ . ..... ^^^^^^^^^^^^^^^^ 

proi^ress toi^ard goal accoKplishment is \ri$ible and ^observable in all btst four* 

. ■• ^ ' . ■ • , : • ^ < V ~ : ^ ; 

Prospects for Permanent ttapact # 



0 



Despite thls^ impressive record <ff progresst a tougher question must be raised: vis. * 
how likely is it that the project teaess* activity will result la parwaneitt- chanKes In ^ 
faculty evaluation procedures? The correspondence between short** and long^tetr ::hinf:? 

I -r ^ ... . . . _ ^ ... , 

is not perlect; particularly In an area as highly-rpolitlcizcd as faculty- evaluation » ' 
risk is hl/gb that, .for one reason or another, success nsay be short-lived. The -c'v i lioi 
team therefore analyzed each of the 30 i^*^ ^.tutions, searched for evidence of pev- . ti : 
and sorted tham into three categories according to their prospects for permanent ir.pa/t* 
The categories were defined as follows. Hl^h probability institutions were those in 
which new policies and procedures had been developed, pilot- tested^ .and intpleffiente<^ 
with the full (or nearly full) support of both faculty and key ^Bminist raters* Iterfm n ■ 
probability Institutions displayed significant progress to the pilot-test stage» nnv? 
were able to provide evidence of administrative followthrough and continued work hy 
the team. Low probablliti^ Institutions either; a) were n^t able to put specific: 
pi^oposals together by the Spring of 1979, or b) i^Jied sigalf^ant adsalnlstratlva or 
political barriers decreasing the likelihood ^of s|!cces?;ful Impleinentatinn, Accnrdinr. 
to these criteria, 10 institutions fell into th group, 15 into the •*teedlu;V' 

group, and 5 iato the "low" group. The table on SAus^ fc -lowing page indicates the statis- 
tical relationship between probability of penoanent, impact and orglnal team goals: 



Team Goals 


Hi all 


Fros^cte for Pen^ence 
Medium Lov 


Total 


Develop ne.w system 


7 


5 


.,3- 


i c 


"^ixxm tune" .current -systes 


3 


5 


• 

1 


9^ 


Develop consistent policies 


0 


5 


1 


0 


, Total 




15 


• 

5 


30 












Chi ttquare(X^) « 


5.50; p<.25 







f 



" llieise appears to be very little relationship ber,;een the nature of teams* original 
I (goals) and their XoDg-term j>rp9pect$: oiia 4cf^ not predict the - otk^r , 

:tp a significant degree. Schools intending to deveXop^a c<^let€ily new system were 
Just as likely to achieve a "permanent iiapsct" status aa schools with less^comprc-* 
hensive goals. Another relationship worth examlnins is that between assessed perma- 



nence and type of institution: 










Institution Type 


High 


Prospects for Permanence 
Hediua tow 


Total 


TWo~year ^ 


5 


3 


1 


9 


Baccalaureate 


.2 


4 


0 


.6 


Kasters 


3 


3 


4 


lb 


Doctoral 


0 


5 


0 


5 


Total 


10 


15 


5 


30 


Chi square 




lO.OA; p<a5 







Here the relationship appears somewhat stronger, although still not statistically 
Significant. There is sooe tendency for the 2-year colleges to have better prospects 
for long-tena change than other types of institutimxs, and for oore of the ^'masters" 
institutions to be in^he "low" group than might be expected. All the same, the 
type of institution and the n$tiire of goals selected at that institution appear 

V 

to be only weak eixplanatory factors,, at best. Other characteristics pxayed a much 

- ■ ■ c a ■ ■ 

more significant role as predictors of success, as explained in the next section of 

this report, * 

♦ ■ • 

Major Factors Responsible for Progress 

While sifting through the evideute of progress from the 30 institutions, including 
reports fronj team members, consultants, and SREB stafft and data collected from the 
15 evaluation site visits^ the evaluation team looked for common themes that see.uL 
to discriminate th^ most-successful £v(m the least-successful projects. When finding 
a characteristic common to the **high probability" institutions, for example, "mediuV 
or •'low** Institutions were examined for the lack of that' characteristic. Seven such 
facti^rs emerged from the analysis* Together they go a long ^ay "towards defining the 
elements necessary to insure successful development and implementation of faculty 

9 

evaluation procedures, at least in these 30 colleges. The seven characteristics are 
listed and defined below, in roughly descending order of importance* 

1# Active support and involvement of top-level admin is t rat ors* The influence of, 

this factor is remarkably pervasive, and in fact far outdistances all oDhers irt ^ 

- * « 

importance. Project institutions at which the president or academic vlce-presl- 
dent early voiced their Support, strongly communicated a sense'of need for. 
change, and actively participated in the development of the new system were, 
without exception, the institutions in the "high probability" group. LikewlSvi, 
institutions characterized by a seemingly apathetic administration fell, again 



^vithout escepticn, into th<; **lotf probability" group. So proaounced vas 
Che apparent influence of this factor that even the amount of adsiinistrati've 
st^port correlated mhao^., perfectly with the degree of pro4|lct success. Such 
support took many fotsts* ranging frcm strong presidential directives backed, 
up by Board or trustee resolutions, to the presence of "line" administrators 
as active end working mes&ers of the team. At one college • for ex^ple* the 
president took every opportunity at college-wide faculty meetings to indicate 
his full support of a new faculty ^valuation system. At another institution 
the academic dean worked behind the scenes obtainixig three years of grant 
support so that his administrative staff and faculty could fully explore new 
procedures. The devastating consequences produced by the lack of top adminis- 
trative Involveuffint is e^mplif led by, examining one of the "low probability" 
schools,^ Here* team meisbers coiiacientlously carried out jthelr plan with 
remarkable enthusiasm; they drew upon SBEB resources » enlisted the aid of 

i 

consultants t and kept the college faculty well infors^d o£» and involved in, ^ 
their activities. Their project has suffered t however, frdta oqly half- ^ 
hearted administrative support » and thus &i\e team has faced an alirast insurmount*- 
able barrier. The importance of both strong and visible administrative support 
cannot be overstressed* 

faculty involvement throu^ftftt the project , Hiis was another characteristic 
of all the %igh probability'* ins titiit ions; at each^ the team was expanded^ 
at least temporarily » to include widerVrepresentatlon oif the faculty; open 
meetings or workshops were held periddlcally to keep faculty informed, and 
team members themselves vmdertook a major responsibility to keep their own 
constituents up to date. Finally > in all the "high probability" schools, 
feedback from faculty members was solicited and responded to, both on a formal 
and iuforoial basis. While, as noted above (point 1), faculty involvement 
is no guarantee of success, it clearly appears to be a necessary copditlon. 



Indeed, severAl iofttltutioiis were iaclitded la^tihe **a|dii« gwhability", group 
by the evaluators even with solid-looking piaafl, primarily because the level • 
of faculty invpivement had been insufficient to generate confidence that 
such plans would last. . 

'S' '■ ■"■'«■ 

0 ....... . . , ' ■ 

Faculty trust in administration. This factor is, of course, suggestfid by 
a combination of factors one and two. Changes in faculty evaluation pro- 
cedures were naich omre likely to be positively received by facility when the 
administration was viewed as responsive to their interests. Such an attitude 
was enhanced when: a) administrators took an active listening role, and 
b) faculty input was incorporated into evolving plans. 



Faculty dissatisfaction with status quo . This factor, and the one following, p.a": 
not be critical to project success on campus; but they certainly give institutions 
having such characteristics a useful push forward. Faculty dissatisfaction vth 
previous evaluation procedures - due primarily to perceived invalidity 
unfairness ~ helped ward off apathy and the usual organizational resistance to 
change.. This factor proved most important in those institutions attempting. to 
"fine-tune" an intact system; in contrast, the lack of this characteristic was 
often a barrier in those colleges attempting to develop something new* ' . 

Historical acceptance of faculty evaluation . One reason, perhaps, why the 
community' colleges generally had an easier time establishing comprehensive 
ev.:u/u™ p.ocea..es was .H. «^3n. no. ... „ .a. „UK .He . J 
'Vhy evaluate?" question. Such expectations are clearly understood at the ^ 
tiipe of a faculty member's appointment. In a similar vein, cotnmunity colleges 

also tend to be the newest institutions, and thus have not built /up academic 

V 

and faculty traditions to the extent of ather institutions*^ An /atmosphere of 
openness and trust in organizational ohange is easier to cultivate*. 

16 
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-i-^ •v^Xuatlon. Many of the projects having serious probleas were not operating 

I under a clear sense of purpose for an evaluation syatea. At other placfts# 

1 \ the formulation of such a policy stateoent appeared to he an event crit«ical 

I - ' ' ■ ' ■ • .■ ■ ■ ■ " ■ • . . 

1 ^ significant gnjgress. At 'on& institution, for exaop^^ 

I was possible onl^ when procedures for evaluation for isqitroves^t purposes 

C'Tortaati'/e") were sharply separated from procedures covering proaotion and 
:*^iawe ("swaaative"). At another, the i^leaentation of a proposed new" formative" 
• » system was delayed by pressures to use the system for "suooative" purposes. 



7« Degree o f centralized institutional decision tiiakinfi . This point is fairly 
selfrrexpl^atory. Consistent policies^and procedures were considerably 
harder to develop in colleges characterised by decentralized detrition foaking 
authority (i.e., a concentration of power at the dean and department chair- 
man level) . - 

For the p^rpose of this project, the above may be considered to be a list of 
"readiness factors ." Colleges fortunate enough to have ousst or all of these factors 
stood to gain the most from SREB project Involvement, and were able to use project 
resources to ,t;heir best advantage. SREB*s role in facilitating development of ^ 
systematic faculty evaluation procedures is assessed in the following secton. ■ 

Role of SMB 



There is little doubt that SREp pljfyed a pivotal role in facilitating accomplishawnt 
of institutional objectives. The/vast majority of individual project teams pointed to 
SREB as a helpful and critical agent; the most often-mentioned comment was, "We 
couLtffi*yhave come as far as we have without SREB Involvepient." There were scattered 
disseffting voices, one for exampla from a college with an already- developed system, 
where the team leader complained about the inapprpprlateness of conference topics; 



ERXC s^ch coOTuents were few and far between, f^y 



sense, the aost Iffiporeant S8EB role was also tbe least subs tan tlve. That is, 
the mere fact of partlcipa^tion in a multi-institutioa project of this sort se^d 
to "spotlight" the inportance of faailty evaluation tm campus, publicize it as aa 
issue, and elevate it to a top-priority status. The existence of a two-year project 
punctuated by periodic conferences helped provide a structure, %rtth goals, activities, 
and oilestones.^ Resultant deadlines helped keep project teams on track. Perhaps 
most important, project participation allowed Institutions to take the time to 
develop a rational structure for development and impXeraentation; in several cases, 
\n fact, participation forestalled hasty responses to extemar pressure from 
Boards j^^^rus tees or system adm.inistrations. 

SR^B impact was not entirely symbolic, howev€'r. Two important cdaponents of the 
.project -Nconferences and campus consultants - each had substantial effects. The 
primary value of the three project workshops was the opportunity they afforded campus 
terns to Wk intensively together with a minimuro of distraction, drawing upon SREB 
resources a^ needed. Interestingly, the actual content of the workshops' as provl4ed 
in the formal sessions seemed most valuable to the teams as a means of identifying 
possible consultants for later use. One other important effect of the woskshops 
should be noted. Many times tesun members expressed a feeling of relief and greater 
self-confidence as a result of interacting with people from other institutions, finding 
that others shared their problems. An implicit goal of the workshops was to allow 
project institutions to share ideas and strategies with one another, and this did not 
happen to a great extent;, |ost teams preferred to work individually, with occasional 
interaction with workshop staff and commiseration with colleagues in other institutions 

The effect of consultant visits to project campuses was much more tangible. In a 
remarkable number of cases, consultant visits proved to be critical, even watershed, 
events, and were almost universally praised by campus teams. Consultants played a 
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variety of roles, all significant. Sometimes they addtsssed the faculty 
as disinterested outsiders, softening opposition, and resistance to diange; sometimes 
they worked with department heads and other administrators, training them to xise 
new. procedures; but most of ten they concentrated their efforts on work sessions with 
the team itself, helloing to define options and focus attention on fundamental Issues, 
such as distlncclons hetweenj "formative" and "sumative" evaluation and the liqiortance 
of system flexibility. In many cases consultants also helped the teaa» anticipate 
problems they wer^ likely to encounter - such as logistical arrangementa for a 
standardized student ratling form - before such prohloas created delays and frustration 



In looking at the overall Impact of SREB across the 30 institutions, the evaluation 
team concluded that at least some of the tah,:^hlg^ probability" schools would 
probably have made significant progress anyway, without SREB involvement; :x their 
Strengths in the seven major "readiness*' factors alone t^uld likely have catted 
them through* (Even^ln these institutions, critical eyents orchestrated by SRSB 
consultants helped make their work niore efficient, and produ&tiv^^t however) # SREB 
probably had the greatest Impact in those schools building new comprehensive systems, 
As suggested earlier, these institutions tended^ to make the most progress, and the 
evidence indicates that the most successful of these utiiTised SREB resources to the 
fullest* 



Lessons Learned 



Statements made by campus team members in vari6u6 self reports during the project and 
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during interviews with evaluators revealed that pVoject participmits have learned a 

\ 

great deal in the past two years of work, not only\ about the mechanics of fae'^lty 

\ * 

evaluation systems but also about the process of^ in^ltutional change. Their comment?? 
have been clustered into six general tuem areas. 



By far the most frequent comment' made hy team s^sib^s was how tine con«mtiii^ the 
.• ' ' ' ' ' * " J " ■ 

whole process turned out to he. The implicatipA is that participants discovered 

how major projects ^ch as these require not only a great deal of effort • hut 

also considerahXe patience. Other related comments were learnings about the 

in^rtance of timing (i.e., fitting the strategy to the institutional climate), 

and the value of "caution." These states^ts all reinforce the e^hasis SREB h^s 

given to rational planning, although perhaps more attention could have been paid 

in the workshops to the value of small but steady incr^^ntal progrpss. 

Another class of team cosssents revolved.. around vliat team oiembers had learned about 
their own faculties. In many cases these « dealt with certain myths atK)ut faculty 
attitudes that were later proven false, aIi of the following io^licit hypotheses ; 
about faculty attitudes were later disproved on at least two campuses; 4) faculty 
members prefer to evaluate one another (versus having the chairman evaluate them); * 
b) faculty members are basically antagonistic to any form of evaluation; and c) 
faculty members are generally vteXl aware of th? system currently used at their 
institution. At the same time, other iugjlicit hypotheses were l«^er supported, such 
as the need for perceived personal is5>act as a prerequisite for faculty support, 
and a generally high faculty interest in (if' not .support of) student ratings of 
instruction. ^ 

'Another whole class of learnings dealt with the politics of organizational change. 
Taken as a group, these statements help reinforce the need for w>rking within"1tbe* 
institution's political systero.. a) "One to one" communication with influential 
faculty members is critical to generating widespread support, h) Faculty evaluation 
is easier with a genuine administrative commitment. ,c) Faculty members must share 
^\ sense of need for change, d) Seemingly simple alterations in instruments or 
documents may make a significant psychological difference to^ the individual being 
evaluated, e) While faculty tend to, be more Interested and acceptant of faculty 
evaluation after some experience with it, ^they are oftlen reluctant to consider 



other viewpoints once they've invested their tioe, f) The etattis (both fomaX ; ' 
asad informal) of teea aseahers is just as iD^orta&t ai their repreieentetives of 
the ca^s cooammity, g) Faculty evaluation von*t «ork if it is slowly is^osed ' 
from the "top dovn"s faculty involvement is critical. . • . 

Severer statements referred to coimection between faculty evaluation ami develop- 
ment, and connections to student learning. Most coaB8>n was a coma^t recognizing 
the ii&portance of tying these concepts together, but expressing frustration that; 
a) faculty development programs are difficult to impien»nt without outside funds; 
and b) relationships to .student learning are extremely di'fficult to document 
asi4 s^sura. ^ 



A fifth .cxvstet of comments contained observations rt:! rating to problexos of instru*- 
mentationy^ primarily that team members realized the limitations^f quantitative 
measurement in some areas, and the difficulty of achieving a consensus on- ^ 
proper weights assigned to evaluative criteria. 

The final cluster of statements - second in frequency only to tlu^se dealing with 
tisoe demands - dealt with learnings about imp^-ementation and manages^t of' 

evaluation systems. Several teams noted some surprise that is^lementai^ion of 

* . " # 

procedures proved to be a more difficult task than the design of such procedures. 
Others noted the^ ismortance of the chairman^ s rtole in evaluation and the need 
for chairman training in this area, the need for continued attention to the 
distinction between "formative" and "summative" purposes for collecting evaluation 
data, and the need for frequent reassessment and evaluation of the new procedures. 



ISw vast a^jprity of p^ject schools (26 out of 30) have ^(!« •ig&lflcant pcogvass 
toward aecostplishoftttt of their original goals, l^ general, iastitution* irlth tlie 
fsoat aabitious objectives (such as building an entirely net^ evaluation systeo) 
displayed the greatest progress* Ten of the 30 indivi^dual projects were Judged to 
have a "high probability" of long range intact, fifteen others fell into a 
Wfflpderatc probability'V category, and five were judged to have only a •'low probability 
of long tem.succ^esa. . PjjospRcts for ii^act appeared'^only slightly related to the 
type of ..institution and the nature of team goals; seven other characteristics* 
headed by the level of support and Involvement by top-level administrators t displayed 
much stron^et relationships. The role of SSEB as a f a(;illtator of institutional 
team goals was highly significant. 'Vhile much of SR£B*s is^act was symbolic <i.e, » 
**8potli^ting*' faculty evaluation on can^ur Ind creating a structure for team 
neoibem), specific project activities - especially eai^us visits by SRE^fKinsomd 
consultants - provided needed input. Finally, project participants appeared ^to 
learn a greaL deal, not only about faculty evaluation, but also about the politics 
of change in. higher education.. 




Data' gathered by the evaluation teaoi have suggested -a nunber of 
eoneIu8iona» related not only to the success. of the SREB Faculty Evaluation 
froject* but also to the^ probable success of similar efforts in other tattings. 

^ Five such conclusions have ^serged from the anaLysiGa 

■* . ** 
1. In order for any faculty evaluation scheme to i»>rfc, four najor con* 

■ • , , . 

ditions must be present. These conditions were found in all. ten of the 
Institutions having the highest probability of penstanent 'project impact* and 
were generally lacking in the less-successful institutions: 

a. Strong administrative support either from the institution's 
president or chief academic of fleer. is absolutely necessary if a college 
faculty, administrative staff, or board hopes to make changes in this area.. 
Where top-level administrative support exists, faculty evaluation enjoys a'* > 
high priority, time is freed up to deal with it, and there appears; to be a 
greater congruence between actual and perceived faculty reward structures « In 
contrast, lack of administrative support is related to complaints -about "lacteof 
time" and faculty grumblings about "academic bureaucracies" and "administrators ' 
hidden motives." This is probably the single most important factor in the 
entire development and implementation process. f 
' • ^ b. Full and extensive faculty involvemsnt is essential* Faculty 

membei;s must feel that the system Is theVrs and that they have hud some part in 

■.J 

its desi%n. Such involvement may be achievcTd in a variety of ways: expansion 

of on-campus teams, frequejft discussion in division or department meetings, «pen 

'' ♦ * , • 

forums, and pilot tests of new systems with the total faculty participating. 



/ :: : p. Thtrs sust.ti%« of ca^tnite tlut f^^ulty and AMBi«» 

tratiott cm draw t^on in devsloplng or revi«ins thnit syst^. Thii npertlae 
can cm froa 4 variety of sources, both latemai aad axtamal to the 
Ifietitutioii, External reaourcea in the torn of coaaultaata play a key role; 
the »ost auccesaful iaatitutiona were thoae at irfiich teas sKBbera mre ehle to 
specify l»w a coaaultaot could best conplement their own atreogtha. 

d. Finally, a generally recognised need for ch«ige in the faculty 

«... 

mluation ayatea must eKiat on caapus. In geiwral, this aeant the presence 
of an unpopular system at the beginning of the project. Faculty meabers nust ' 
feel the^ have something to gain by change; at institutions i^ere faculty 
r^ers eeeaed apathetic or complacent, project teams generally had trouble 
getting' started. 

2, the kind of regional, interinatitutional approach used by SREB in this 
project warrants consideration as a model for dealing with other major issues 

f 

in higher education. Twenty-six of 30 project ,teaas demonstrated significant 
progress in eighteen month's time, and 25 of these showed at least a moderate 
prognosis for permanent impact. A regionally-coordinated effort, with its 
conferences, deadlinaa, and reports, provides invaluable outsit stimuli to 
the institution, and helps to elevate the status of project activities on 
campus. This appeared to be true even of the participating institutions who 
would likely have made significant progress without SREB' s help. 

SREB staff members made important contributions, and were critical to the 
success of the project as s v^ole. They communicated frequently with partici- 
pating colleges and their campus team leaders. They planned and implemented 
workshops for team metiers fro^ff all 30 schools, and gave them an opportunity to 
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work together and get to know .one another, they ,hei{»ed colleges select 
consultants » while serving as informal consultants and catalysts themselves. 
In short* they Kept things moving. : . 

• . . . ... ' ' ■ . ^ ' ^ - , , , ^ ... , ? , , ■ , A*' 

3. Another Ij&portant; conclusion- inferred from the ,$valua(ion data Is that 
no ane faculty evaluat^ion system is necessarily better than any other* There 
was a tremendous diversity of culture, tradition, and norms among the 30 proiect 

• - % * ^ * ■ i 

schools^ and clearly ^ what worked at one institution (faculty growth contracta^ 

for exas&ple}| ^euld not have worked at another. Campus tej^iQ members seemed 

sensitiire to this fact, and there were few, if any « attempts to borrow whole 

systems from other institutions. Rather, the sharing which took place during 

♦ 

the regional coafercnces tended to be u. sharing of common problems and 
occaslohally, spec|;flc Instruments. Successful systems were developed from 
within, not imported from somewhere else. 

4. Most of the participating colleges appear to^havc made a direct and 

visible tie between faculty evaluation and faculty devj?lopment ♦ The ties took 
X 

mjsny forms: career development plahs, new grant proposals for faculty develop- 
ment^funds, plans for new or upgraded faculty development .of flees, and so forth* 
Many professlos^als in ehe field of faculty development and evaluaelon have taken 
the -position tbak development and evaluation prcgramfs Dugbt Ko be kept separate. 
Observations derived from the project schools , however, suggest that such a 
separation way not be necessary and my evt?n hinder the successful revision ,of 
'an institution's faculty evaluation program. Often the successful occcp*-Ance 
of a new or revised f^iculty evaluation systern appeared tp b^ enhanced by an 
administrative commitment to do more in faculty ilcvciopncnt . 

"7 5. Most project participants voictfd an apprecintion of the logical 
connection between facMlty evaluation ami jstudent learninf,. But in only one 



Isistltutioii was osi, attempt «ade (even on a' saall scale) to jitu^ the sielatiosi- 
^shlp esplrically. The 30 participating colleges redevsd yeiy little assistasice 

' . ■ » «* ■■■>■.■ • • 

« ' » • .' 

£na SREB in atteoftlng to nake this connection* Not o^y colleges had teached 
the point of even considering student leatning seriously as a means o£ oeasuringl 
teaching effectiveness. Those fev that had were unable to mo^e further; 

.In suomary, the Faculty Evaluation Project has dentmstrated that significant 

■ ■ \ • •• ...... ^ ■ . . .... ^ ^ 

results are achieyahle with a regional* multi-ins tl^utional approach at a 
reasonably low cost, if four characteristics Cadslnlstratlve support* faculty 
lovolvementt base of vexpertiSie> ai(d rec^^ change) ara present in 

the individual ins titutiph^* Without these cliaracteristics, prospects for 
success are low* no «^ttef what SREB dof s; but if these characteristics are 
. sufficiently scrongt f^articipation in a m project like this one 

has a great deal to offer. , 

* - ' ■ •''^ 

♦ 

■ I ■ * 




The purfioAe of ;thi» £ocm is to give your team an <^i»ort unity to convey to 
the SREB etAfC an^ your consultant your reactions to and constructive criticissi 
of the cons|b.tant*s «ork with you. It is intended ^that the ompleted form repre- 
, sent ths concensus of the team* Please complete and mil this form in to ViXlism - 
O'Connell, Director, Faculty Evaluation Project* Southern Regional Education Board, 
130 Sixth Street, N,W., Atlanta* GA 30313, no later- than two vedcs after your con- 
sultant's visit, * 

tart I - General Backgrotmd Information 

.1, Name of your College ■ ■ 



'2. Name of your Consultant 



3. Dates of the Consultant's Visit 

Part II - Your Objectives 

' Pirectionsi Please list the objectives you had for inviting a consultant to yoyr 
cacqius and then Indicate the ext^it to which these objectives were sset. 

Response Keyi (Circle appropriate response for each Objective.) 

- 1 * Not Met^ 2 - Partially Met 3 - Largely Met A - Fully Ifet 

States^nt of Your Objective is) 



2. 



3 



3- . 



Comments or Additional Objective^: 
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''■4'. 



■ > , \ - ■ ...V . . ■ • • . ■ ■ 1 ■ t .' ■ 

\ ■ • • . . , , ■■ ^ < 

■ '\ - ■^--'* . ' . - ■, ■ .- 

Part III " C^ticaX Events during CopgtU.tant'g Vtait . . 

Directionai deseribe any critical events that occurred during the con- 

sultant's visit and indicate ultether they iielped or hindered the accoi^lishinent 
of the above objectives. 



n 



Part IV - Anticipated Qutcoaaes 

Pirectionsi Idat below the outcosaes ypu think can be esqsected as a result of 
this consultation visit. 



I 
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Part V - Ovarall Bvalitatiwt of Coagaltant 

Directions; Fi«a«e AvaXjVMite the followias characteriei^ics ol your consultant • 

Responaa Keys x thiable to^^u4ge 2 - Poor 4 • Very Good 

I VftSy Poor. 3 ■ Average or Fair 5 « Excellent 

1. £££ectl\rat . interpersonal skills x 1 ^ . 3 , 4 5. 

2. Knouledge o£ the field of Faculty Evaluation . x 1 ^ 3 4 5 

3. Knowledge of the field of Faculty j)eyelopment ^1 2 3 4 5 

4. Knowledge ^f the field of Hi^^er Education s I 2 3 , '4 5 

5. Problem solving skills x .1 2 ^ 4 5 

6. Helpfulness to our team x 1 2 3 4 5 

7. Knowledge of our institution xl2345 
Comments; 



ftart VI - General Coisaents 

Directions: Please use this section to give U3 any additional consnents^ suggestions » 
or observations about your consultant's visit or how this Faculty Evaluation Project 
could be 8K>re successfulo 



SIGNATURES OF T£M HES®£&S; 



Thank you for your assistance in completing this .^valuatipn form* Your assistance 
is deeply appreciated. Please make a copy of^thi^ completed form for your files 
(evaluation portfolio) and then return the form within tvo weeks after your con- 
sultant's visit to: William O'Connell, Director, Faculty Evaj.uatlon Project, 
Southern Regional Education Board, 130 Sixth Street, N,W. , Atlanta, GA 30313. 



Southeni Begional Edueation Boac4 
; f acuity lEvaimtioB 

GQHSULXABI'S ZVAXmnm OF G01i5m.IAXIQ!i iNQ TEAM P80GBESS 



The purpot* ol^ thit form is to givt you m oppoxttmity to c0Bi9»f to die SUB 
«taff and to tha pieojaet teairyour raactioim to and conatructive critidaa of the 
team's vozk with you, ?laaae coaplate and nail this fom to William O'Omnall, « 
Director, Faculty Evaluation Frojact, Southern Begiooal Sducatlcm Board, 130 Sixth 
Street, N.W. , Atlanta, Georgia, 30313, no later than tvo weeks after your consulting 
visit, . . . 



Part I - General Backgroimd Inf oraation 
1. ^ux Nana 



X, 



2. Naae of College Visited 



3« Dates of Consultation Visit 



Fart XI - Your Objectives 



Directions i Please list the objectives you had for your consultation visit and 
then indicate the extent to which these objectives were laet. 



Response Key; (Circle appropriate response for eadi objective.) 



*> 1 - Not Met 2 - Partially Met 
Stateioent of Your Objective(s) 

1. . L 



3 - Largely Het 



, 1 



2. 



4 - Fully Met 



3. 



Conm^ilta or Additioiial Objectives? 



on Back) 



--■.v,.%..i. . 



Part III - Critical Evepta 




iPtrectlooai Please list below any critical events that occurred ei^er p^rior to > 
; or during yotif visit and indicate whether they helped or hindered the accoinplish- 
n»nt o£ the above objectives. * ■ . . . 



■ • 0 
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gayt IV « Anticipated Outcoaee aa a Reault of Your Vi«it . . 

Pirectlong! ilst below the aatlcipated outcoiaea you think lean be es^ected aa a 
xesult of your work with this college and ita. project teate. 



Part V - Perceived Resources and StrenRtha/Barriegfi and Weakneaaea 

Directions; Please describe briefly any resources or strengths you perceive to 
-^e helpful in this team's progress to date. Also« similarly describe any 
. perceived barriers or weaknesses that isay be hindering progress. 



'IS- 



o 

ERIC 
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Part VI - Your Btwlmtiop of TiMa 
Oiractlow AVftXiiat* tiM t««m you tiorked wltli on the foUowlBg itcas: 



Regpongft K«y: 



S • Exe«ll«at 
4.*'' Very Good 



3 * Avarsgo or Taix 
2,« Poor 



1. team** progrus to date ' * s 

2. Cohealveneas of tbe team x 

3. Level of coBsmnlcatlon aaong team aeml^ers x 

4. Quality of intexptraonal relatiooships on 

the team ■. Jt 

.5. Team's skill In tcnowlng how to uae a 

. coaaultant x 

6. Quality of plaimifig for conaultaat'a Arisit % 

7. Use of consultant* a ti8» i^ile on caiqiua x 

8. Team' a ability to aet objectives x 

9. Chancea that this team will effect aignifi* 
cant changea in the faculty evaluation syatem 
at their college x 

10, Team's ability to aaseaa Ita own progreaa x 

, Comments: ' - 



1 
1 
1 



1 
1 
I 
1 

1 
1 



1 • Very Poor 

X - Unable to Judge 



2 
2 
2 



2 
2 
2 
2 

2 
2 



3 

3 
3 



3 
3 
3 
3 

3 
3 



4 

4 
4 



4 
4 
4 
4 

4 
4 



5 

5 
5 



5 

.5 
5 
5 

5 
5 



Part VII - <^neral Commenta 

/ 

Directions t Pleaae uae this section to 'give va any other additional cotmentSf 
suggestions or observations about your consulting visit or about how we could 
make this Faculty Evaluation Project more succesaful. 



Date 



Consultant's signature 



N THANK YOU FOR YOUR ASSISTANCE IN COMPLETING THIS EVALUATION FORM. YOUS -ASS I STANCH 
IS DEEPLY APPRECIATED, Please return this conipleted form w'ithin two weeka after 
^ your visit tos l^illiam O'Connell, Director, Faculty Evaluation Project, Southern 
ERIC Regional Education Board, 130 Sixth St., NW, Atlanta, GA 30313. , 



<t«s9lA Com from one c£ thre« vorkshopt) ' 

TOSKSHOP FORM A 



SREB FACULTY EVALimtlON mJECr, WORKSHOP JANUARY 22-24, 1979 

EVALUATION mm * 



The purpose of this form Is to give each participant an opportunity to convey 
to the workshop staff and project ovaluators his or her reactions to the final pro-- 
Ject Workshop. Please leave youk compXe ted fom with an SREB staff nes^er before 
leaving the meeting. « \ , 

Part I - Your Objectives 
(To be Partially Cbn^leted Curing the First Workshop Session) 

Directions; We are interested in knowing of your si^ecif ic objectives for this 
workshop. Please list your objectives at the beginning of the workshop, and 
then at the end of the workshop rate how well these objectives were met. 

Response Key i (Circle appropriate response for each oi^ective) 

1 « Not Met 2 f Partially Met 3 - Largely Het 4 - Fully Met 



Statement of your Objective (s) 
1. • 

. 1 2 3 A 



2. • 

1 2 3 A 



'■^ ^ : 

'_ 1 2 3 A 

CoTmnents or Additional Ob;lectlves; 
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— ^art II - Workshop obje^^ , 

Directions t Please indicate how well you think the following workshop objectives 
were wet, twlog the same response categories as above (l«Kot met; 4>F<illy met), 

I* Team assessed Its progress to date " i 2 3 A 

2. Team identified problems/barriers and ' ' ^ 

solutions to same 12 3 4 

3, Team refined its plans for improving 

faculty evaluation 12 3 4 

4» Team made effective use of resource 

consultant (s) at the workshop . 1 2 3 4 

5. ^ Team defined procedures for gaining final 

acceptance of 'new or reVioed evaluation system, 
: and for implementing nw/reviaed system 1 2 3 

6, Team gave attention to malntaininf? momentum and 

continuing progress .beyond June 30, 1979 12 3 4 

. "" * « 

7i Team/ individuals had helpful interaction -with 

workshop participants from other Institutions - 1 2 3 v 4 

If so, fjrom which institutions? 



Comments : 



e ■ . 

Part III - Evaluation' of ^sources 

Directions; Please indicate your overall rating of each of the following items: 

Response Key; 1 - Of No Help 3 « Helpful 

2 - Of Little Help 4 « Very Jfelpful 

1. Resource consultants 12 3 4 

2« Workshop resource center (display oaterials, 

message centers contact point) 12 3 4 

3. SREB staff (as conference faciUtatprs) * 1 2 3 4 

« 

Comments ; • o i 



'■V 



, 1. Uhlch workftl»9 aessioaCs) or actlvlty(ieli) did you find thu sost uctful? Why? 



J 



2. Which 8esslon(s> or activltyCles) did you find to be leaf t useful? Why? 



Coionehtfi: 



^ Part V - Evaluaticm of Workshop Fomat: and Arrangetaents 

ft 

Directiona: In order to h^lp us in the planning of fitture vorkabops like this 
pne, please indicate your evaluation of the foUQVlng aspects of the vorfcahop. 



- Response Key? 

X " Unable to Judge 1 ■ Poor 2 - Fair 

1. Workshop location 

2. Food services 

3. Workshop physical facilities' 
4« t^rkshop schedule/agenda 
Coaaaents: 



3 - Very G^od 4 • Excellent 
X I 2 3 4 



X 
X 
X 



1 
1 
1 



2 

. 2 
2 



3 
3 
3 



.4 
4 
4 



Pleafe use the back of this page for additlcmal cosmients on any aspect of the 



ERXC workshop and/or project fomat imd approach. ^^^'^'^ y^^* 



